---

HTTP Detection Techniques

Detecting Threats in HTTP Traffic: A Comprehensive Guide

Following on from analyzing DNS traffic for threat detection, the next focus is HTTP—another core element of internet communication. Why DNS and HTTP? Imagine trying to navigate the web without them; it’s nearly impossible. HTTP, the protocol enabling communication between web browsers and servers, is everywhere, especially in web applications, And web applications are everywhere :D. making it a valuable source for network security monitoring.

This article provides an in-depth exploration of using HTTP traffic to detect malicious activity, presenting a variety of techniques and real-world examples to help you defend your environment proactively.

---

Why HTTP for Threat Detection?

Given its prevalence, HTTP traffic serves as a treasure trove for security monitoring. Many cyberattacks leverage HTTP in some form, allowing malicious actors to blend in with normal traffic while compromising systems and exfiltrating data. By scrutinizing HTTP traffic, we can identify unusual patterns and respond to threats promptly.

HTTP Traffic Monitoring for Inbound and Outbound Threats

HTTP analysis can reveal both inbound attacks (originating from external sources) and outbound threats (potential malicious behavior from within the network). Identifying the direction of the traffic is critical in understanding intent and tailoring defenses.

• Inbound HTTP threats often include attacks such as:

  • Brute-force attempts
  • SQL injection
  • Cross-site scripting (XSS)
  • Directory traversal
  • Remote code execution (RCE)

• Outbound HTTP traffic might reveal:

  • Data exfiltration attempts
  • Command and Control (C2) communications
  • Malware downloading additional payloads (staged malware)

Key Log Sources for HTTP Threat Detection

To effectively monitor HTTP traffic, it’s essential to gather logs from different sources, depending on whether you’re analyzing inbound or outbound traffic:

• Inbound HTTP Logs:
  • Web Application Firewalls (WAFs)
  • Web servers (e.g., IIS, Apache, Nginx)
  • Intrusion Detection Systems (IDS)
  • Load balancers
• Outbound HTTP Logs:
  • Proxy servers (e.g., Squid)
  • Network firewalls
  • Endpoint Detection and Response (EDR) solutions
• Specialty logs for HTTP Analysis: Zeek, PacketBeat, custom scripts, and cloud logs/APIs, among others, can provide valuable insights. For proxies, consider utilizing the X-Forwarded-For field to identify the original source IP.

---

Enriching HTTP Logs for Deep Threat Detection

HTTP logs can be massive and overwhelming in their raw form. To turn this data into actionable insights, log enrichment techniques are key:

• Field Length Analysis: Monitor the length of HTTP fields like User-Agent strings and URLs. Abnormally long values often signal reconnaissance or buffer overflow attempts.
  • Example: Requests with excessively long User-Agent strings (over 200 characters) revealed an attacker probing the web server’s configuration.
• Tagging and Labeling: Use tags to categorize HTTP requests by attributes like IP address, ASN, response code. This simplifies filtering and further analysis.
  • Example: Tagging requests from known Tor exit nodes as “anonymous” helped quickly identify suspicious activity associated with vulnerability scanning.
• String Randomness Analysis with freq.py: Utilize tools like Mark Baggett’s freq.py to detect randomly generated or obfuscated strings. This is useful for identifying Domain Generation Algorithms (DGAs) or unusual User-Agent strings in HTTP requests.
  • Example: Analysis of the Host header using freq.py flagged requests to randomly generated domains, pointing to a DGA-based malware infection.

---

Practical Techniques for HTTP Threat Detection

Here are some key techniques for identifying malicious HTTP activity, supported by slingshot THM's room scenarios:

1. Threshold-Based Anomaly Detection for Request Rates (INBOUND):
   • Concept: Monitor the number of GET/POST requests per minute from a single IP address. Unusually high request rates often indicate brute-force, DoS, or scraping attempts.
   • Example: here you can see 2507 GET record only from one ip.

2. 404 Error Monitoring (INBOUND):
   • Concept: A high number of 404 Not Found errors often points to vulnerability scanning or directory traversal attempts.
   • Example: A single IP generating 1800 404 errors in 30 minutes revealed a vulnerability scanner probing common attack paths, like /ad8min and /cgi-bin.

3. Anomaly Detection for 200 OK Responses (OUTBOUND) :
   • Concept: Excessive 200 OK responses, especially paired with large data transfers, may signal exfiltration or successful brute-force attempts.
   • Example: A surge in 200 OK responses with over 1GB of transferred data from an internal IP indicated unauthorized data exfiltration.
4. Anomaly Detection for 200 OK responses (INBOUND) :

here you can dominant count of 200 responses by 10.0.2.15

5. Detecting Naked IP Addresses in HTTP Requests (NOT INCLUDED IN THM ROOM):
   • Concept: HTTP requests with direct IP addresses in the Host header instead of domain names often attempt to bypass security mechanisms.
   • Example: A series of requests with IPs in the Host header targeted an internal server, bypassing domain-based firewall rules.

2. URL Length Analysis (NOT INCLUDED IN THM ROOM, RECOMMENDED ENRICHEMENT):
   • Concept: HTTP requests with direct IP addresses in the Host header instead of domain names often attempt to bypass security mechanisms.:
   • Concept: Long URLs often signal attempts at buffer overflow, XSS attacks, or data exfiltration.
   • Example: A request with a URL over 5000 characters, containing an encoded string, revealed sensitive customer data being siphoned out.
3. User-Agent String Analysis:
   • Concept: Unusual or impersonated User-Agent strings can expose bot activity or outdated, vulnerable browsers.
   • Example: in this case can easily identify major used enumeration tools.

Conclusion

Analyzing HTTP traffic is fundamental for detecting and countering a range of malicious activities. By enriching HTTP logs and applying these detection methods, you can gain deep insights into your network activity and proactively safeguard your environment. Remember, security is an evolving process—continue refining and adapting your monitoring techniques to stay ahead of ever-evolving threats.